An Extreme Point Tabu Search Method for Data Mining

We propose an Extreme Point Tabu Search (EPTS) algorithm that constructs globally optimal decision trees for classiication problems. Typically, decision tree algorithms are greedy. They optimize the misclassiication error of each decision sequentially. Our non-greedy approach minimizes the misclassiication error of all the decisions in the tree concurrently. Decision trees are ideal for data-mining because their logical structure makes them easily understandable. Using Global Tree Optimization (GTO), we can optimize existing decision trees. This capability can be used in data mining for avoiding overrtting, transferring knowledge, incorporating domain knowledge, and maintaining existing decision trees. Our method works by xing the structure of the decision tree and then representing it as a set of disjunctive linear inequalities. An optimization problem is constructed that minimizes the errors within the disjunctive linear inequalities. To reduce the misclassiication error, a nonlinear error function is minimized over a polyhedral region. A new EPTS algorithm is used to search the extreme points of the polyhedral region for an optimal solution. Promising computational results are given for both randomly generated and real-world problems.